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Abstract 



As colleges educate a more diverse and global student population, there is increased need to ensure every student 
succeeds regardless of their differences. This paper explores the relationship between equity and assessment, 
addressing the question: how consequential can assessment be to learning when assessment approaches may not be 
inclusive of diverse learners? The paper argues that for assessment to meet the goal of improving student learning 
and authentically document what students know and can do, a culturally responsive approach to assessment is 
needed. In describing what culturally responsive assessment entails, this paper offers a rationale as to why change 
is necessary, proposes a way to conceptualize the place of students and culture in assessment, and introduces three 
ways to help make assessment culturally responsive. 
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Equity and Assessment: 

Moving Towards Culturally Responsive Assessment 


Erick Montenegro and Natasha A. Jankowski 


Introduction 


College enrollment has become increasingly diverse in terms of students’ 
race, ethnicity, gender identity, socioeconomic status, sexual orientation, 
age, ability, etc. This trend is only expected to continue as the United 
States moves into a majority-minority nation by the year 2050, and college 
enrollments continue to increase. Conducting assessment in a manner that 
takes into consideration the various needs of different student populations is 
a responsibility of higher education. For one, underrepresented students are 
more likely to be low-income and first-generation (Del Rios & Leegwater, 
2008; Li & Carroll, 2007; Benitez, 1998), and there are vast differences 
between the needs of lesbian, gay, bisexual, transgender, queer, intersex, asexual 
(LGBTQIA) (Check & Ballard, 2014; Mallory, 2009), undocumented (Kim 
& Diaz, 2013; Perez, 2010), nontraditional (Macqueen, 2012), and special- 
needs students attending higher education institutions (Froese-Germain & 
McGahey, 2012). Further, students are increasingly mobile, with transfer 
students coming from mostly traditionally underrepresented backgrounds, 
attending multiple institutions (Backes & Velez, 2015; Shapiro et al, 2012) 
and facing their own challenges in higher education (Tobolowski & Cox, 
2012). 

Various areas of higher education are aware of the need to accommodate 
different student populations because “individual differences are clearly 
important to student success” (Strange & Banning, 2015, p. 61). For 
example, approaches to teaching, student development, student services, 
and campus programs have been analyzed and altered to improve outcomes 
for specific student groups (Ladson-Billings, 1995a; Ladson-Billings, 1995b; 
Schuh, Jones, Harper, & Associates, 2011; Kezar, 2011; Lara & Wood, 
2015; Bowen, Chingos, & McPherson, 2009, Aronson & Laughter, 2016). 
Within the field of campus advising, the issue of microaggressions through 
lack of cultural awareness has been raised (Chu, 2016) and the work of the 
Association of American Colleges and Universities (AAC&U) in their Equity 
Imperative outlines the need to understand who students are, disaggregate 
data to look for inequities, and explore policy changes for unintended 
impacts on student groups. Conversations in K-12 have addressed the notion 
of equity from the standpoint of equity traps (McKenzie & Scheurich, 2004) 
within schools and the need to prepare school leaders to not only expose but 
address them through courageous conversations about inequities (Singleton, 
2012). In a literature review of culturally responsive school leadership, 
Khalifa, Gooden, and Davis (2016) argue that culturally responsive leaders 
need to continuously support minoritized students through examination of 
assumptions about race and culture. Further, they argue that as demographics 
continue to shift, so should practice that responds to student needs, finding 
that it is “deleterious for students to have their cultural identities rejected in 
school and unacknowledged as integral to student learning” (p. 1285). 


Conducting assessment in 
a manner that takes into 
consideration the various 
needs of different student 
populations is a responsibility 
of higher education. 
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Culturally relevant and culturally responsive pedagogies sought to outline 
ways in which teachers could address unique learning needs of diverse student 
populations. Gloria Ladson-Billings (1995a, 1995b) recommends using 
culturally relevant pedagogy as a way to allow students in populations outside 
of the majority to maintain their cultural integrity all- the- while succeeding 
academically. Culturally relevant pedagogy aims to “produce students who 
can achieve academically. . .demonstrate cultural competence, and develop 
students who can both understand and critique the existing social order” 
(Ladson-Billings, 1995a, p. 474). In culturally responsive pedagogy, teachers 
use aspects of students’ cultures in an asset-based approach as opposed to 
deficit-based to make the course material relevant to them, and increase 
their skill acquisition, engagement, and learning outcomes (Ladson-Billings, 
1995a). Yet, Geneva Gay (2010) has argued that solely modifying teaching 
practices cannot solve the challenges faced by minoritized’ students. 

In terms of assessing student learning, the field has been largely quiet when 
it comes to issues of equity. Assessment, if not done with equity in mind, 
privileges and validates certain types of learning and evidence of learning 
over others, can hinder the validation of multiple means of demonstration, 
and can reinforce within students the false notion that they do not belong in 
higher education. For equity gaps to be addressed, an entire institution needs 
to explore the combination of solutions and supports needed for students 
to be successful (Jones, 2015; Methvin & Markham, 2015), of which 
assessment is one. However, little of the conversation thus far has focused on 
the connection points between demonstration of student learning and issues 
of equity. Instead, assessment has remained largely unchanged in regards to 
inclusivity, and little urgency has been given to ensuring that students are 
provided with just and equitable means to demonstrate their learning. There 
is a difference between assessing all students in the same way in relation to a 
specific outcome of interest and making sure assessments are appropriate and 
inclusive of all students. Being attentive to how students may understand 
questions, tasks, and assignments differently, as well as feedback regarding 
their learning, is not only beneficial to students but to internal improvement 
efforts as well. Intentionally choosing appropriate assessment tools or 
approaches that offer the greatest chance for various types of students to 
demonstrate their learning so that assessment results may benefit students 
from all backgrounds advances our collective interest in student success. 

Without examining issues of equity the students who may stand the most to 
gain from assessment efforts may have the least benefit since their learning 
is not accurately assessed and feedback may not be relevant to impact 
learning. If assessments are to be holistic in their goal of improving student 
learning, then incorporating a culturally responsive approach to assessment 
is a priority. As C. Carney Strange and James Banning (2015) state, student 
cultures “can play an important role, for good or otherwise, in introducing 
students to and maintaining their engagement in the learning process” (p. 
53). It also creates opportunities for students to experience deep learning 
(Entwistle, 2001) by honoring students’ prior knowledge and experience. 
However, before we present the concept of culturally responsive assessment, 
it is useful to unpack an assumption that hinders consideration of diverse 
learner needs within assessment — that while learners may take multiple 
paths to and through learning, they must demonstrate their knowledge and 
skills in the same way. 



Assessment, if not done with 
equity in mind, privileges 
and validates certain types 
of learning and evidence 
of learning over others, 
can hinder the validation 
of multiple means of 
demonstration, and can 
reinforce within students the 
false notion that they do not 
belong in higher education. 
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Limiting Learning Demonstration 


There is an assumption at play within the field of assessment that while 
there are multiple ways for students to learn, students need to demonstrate 
learning in specific ways for it to count. For instance, in a specific course 
different approaches may be used to engage students in the material, but 
demonstration of a students’ knowledge, skills, and abilities are done 
uniformly in the same assignment or approach — so while there may be 
multiple approaches and methods used across a program or institution for 
assessing student learning, at each instance of demonstration a single approach 
is employed. Regardless of the literature on the multiple ways students acquire 
knowledge, assessment asks students, at each instance of demonstration, 
to show they have the knowledge and skills of interest through the same 
means. William Sedlacek (1994) discusses the need for the development of 
multicultural assessment standards within the Association for Assessment in 
Counseling (AAC). While the focus is upon assessment within the context 
of counseling support and services, the interest of addressing the needs of 
those with “cultural experiences different from... White middle-class men 
of European descent, those with less power to control their lives, and those 
who experience discrimination in the United States” (p. 550), remains the 
same for assessment of or for learning in higher education. Sedlacek (1994) 
identifies five fallacies related to culture and assessment, stressing that most 
measures were not designed with nontraditional or underserved populations 
in mind, that few assessment specialists are trained in developing measures 
for use with nontraditional populations, and that larger issues exist that 
need to be explored and addressed when promoting diversity, equity, and 
inclusivity through assessment. Of note is the fallacy referred to as the three 
musketeers, which is the idea that in order to make a measure equally valid 
for everyone, everyone completes the same measure — all for one and one 
for all — as a means to ensure fairness instead of using different measures 
for different groups. Yet, Sedlacek (1994) argues, “if different groups have 
different experiences and different ways of presenting their attributes and 
abilities... it is unlikely that we could develop a single measure or test item 
that would be equally valid for all” (p. 550); further arguing that there is no 
need to employ the same measure when what is desired is equity of results, 
not process. 

There are institutions providing students with support and opportunity to 
choose from a variety of approaches or even design how they will be assessed 
in cooperation with faculty members, presenting students with agency and 
choice in the assessment process (Singer-Freeman & Bastone, 2016) and 
most institutions use a combination of assessment methods to gauge learning 
(Kuh et al., 2014). In a study at the University of East London, students were 
allowed to choose how they were assessed, significantly improving attainment 
among learners without an academic background (Grove, 2016). Instead of 
completing exams based on coursework, students were given the option to do 
a presentation, poster, or debate. Using the alternative assessment techniques 
“helped mitigate the fact that many first-year students had not been in 
formal education for some time” allowing them space to demonstrate their 
learning, not their exam-taking abilities (Grove, 2016). Further, a similar 
approach was used at the University of Dublin where students were able to 
make a poster instead of taking an exam. In both instances, students had to 
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demonstrate their learning on the same learning outcomes and evaluative 
criteria, but the manner in which they did so was irrelevant. Rubrics were 
used such that the evaluation of the work was the same, thus quality ensured, 
but the demonstration could be different. In Canada, a study was undertaken 
within a large, third-year psychology class regarding differentiated evaluation 
to examine student engagement, quality of learning experience, and address 
challenges associated with increased student diversity (Gosselin & Gagne, 
2014). Differentiated evaluation allowed students to choose how they would 
be evaluated though all students were still required to take mid-term and 
final exams. Students had the option of adding a term project through 
preparing a mini-class or participating in a community service learning 
program. The study found positive impact on student achievement and on 
the learning experience, with students performing below class average seeing 
grade improvement when completing a term project. Further, students who 
completed the project performed better on the final exam in comparison to 
those that did not, and the option helped to alleviate stress of sitting for an 
exam. Qualitative responses from students that selected the project option 
indicated that they saw the alternative as an opportunity to demonstrate 
their learning through a format over which they felt more control. Gosselin 
and Gagne (2014) argued that there are “methods of assessment that can 
foster inclusiveness and academic success whilst upholding high standards 
for the quality of student learning” yet interestingly “most innovations in 
this context have focused on teaching rather than on student learning” (p. 6). 
The differentiated evaluation approach complemented the existing structure 
and allowed the relationship between faculty and student to shift to one 
of collaboration instead of power, regarding decisions about how students 
demonstrate their learning. 

The need to fold in culture and student experience into assessment is stressed 
in the everyday expertise framework — a perspective of learning that takes 
into account how students demonstrate knowledge and skills in their daily 
life with the other people around them (Toomey Zimmerman & Bell, 
2012). The framework allows for learning to have multiple dimensions 
including individual, social, and cultural, requiring a broad consideration 
of how people learn within and across learning environments, noting that 
learners do not act with equal competency in all settings, even if the content 
is the same. Toomey Zimmerman and Bell (2012) argue that the difference 
in performance indicates that learners competent in informal and everyday 
settings may falter in more formalized learning settings, requiring alternative 
means to demonstrate their knowledge outside of the traditional classroom. 

Beyond the many benefits from engaging students in co-curricular 
experiences (Meents-DeCaigny & Sanders, 2015; Schuh, Jones, Harper, & 
Associates, 2011; Schuh, 2009; Pascarella & Terenzini, 2005; Del Rios & 
Leegwater, 2008), co-curricular learning provides a means to address the 
issues raised by the everyday expertise framework by widening our lens of 
where learning happens to include experiences beyond the classroom. In 
addition to conceptions shifting where learning happens, there has been a 
rise in competency-based education (CBE) which releases the time structure 
in which learning occurs in terms of credit hours. CBE programs stress that 
authentic artifacts, or demonstrations of student learning, need to come 
from a variety of sources to engage learners with curricula and assessment 



The manner in which students 
demonstrate learning is 
irrelevent when student 
demonstration is held to the 
same learning outcomes and 
evaluative criteria. 


National Institute for Learning Outcomes Assessment | 7 


that reflect not just multiple ways to learn but multiple ways to demonstrate 
mastery of a competency (Jobs for the Future, 2016). However, there are 
calls for significant research to determine how best to design assessments 
for underprepared learners that also elevate and validate their skills through 
alternative measures (Person, Goble, & Bruch, 2014). While learning may 
happen anywhere and learners may need different lengths of time in their 
learning process, there is still the issue of who gets to validate that learning 
has occurred, or that demonstrations of learning are of the ‘right type.’ 

While there is movement to more inclusive means of assessment and active 
engagement with students as partners in learning, it is clear that the challenges 
of various minority groups on campus differ from those of the majority 
(Ellis & Chen, 2013; Rankin, Weber, Blumenfeld, & Frazer, 2010; Miller, 
Bradbury, & Pedley, 1998), yet higher education still privileges certain types 
of learners, certain ways of demonstrating knowledge, and certain learning 
spaces by not consistently offering transparency, differentiated assessments, 
or empowering students in their own learning. Students need to develop 
and apply their knowledge and skills across multiple contexts in different 
courses through a range of methods (Newman, Carpenter, Grawe, & Jaret- 
McKinstry, 2014, p. 14) with integrative liberal learning requiring students 
to engage in “ongoing demonstration to themselves and to others, of the 
gains made through curricula, programs, and the educational experience 
as a whole” (Ferren & Paris, 2015, p. 5). Yet, the signals education sends 
to students about what is validated or counts as demonstration of learning 
can be detrimental and reinforce for marginalized students that they do not 
belong because their learning ‘doesn’t count.’ What is needed is collaboration, 
where students, faculty and staff “draw together their life experiences and 
aspirations with classroom, co-curricular, and community opportunities” 
(Ferren & Paris, 2015, p. 20). 

Culturally Responsive Assessment 



While learning may happen 
anywhere and learners 
may need different lenghts 
of time in their learning 
process, there is still the issue 
of who gets to validate that 
learning has occurred, or that 
demonstrations of learning are 
of the ‘right type . * 


Defining “culture” and explaining what is meant by culturally responsive 
assessment is complicated. The issue is that culture, whether speaking 
about it in terms of an organization, a campus, or an individual, has been 
historically difficult to define. Higher education has a tendency to group 
student differences and issues around race under the term ‘diversity,’ which 
is often discussed in relation to benefits to White students as opposed to 
African Americans, Latinx, Asian Americans, and Native Americans who 
continue to be underrepresented in higher education (Dowd & Bensimon, 
2015). While diversity efforts on college campuses have brought attention 
to the vast differences among students — including gender, religion, sexual 
orientation, etc. — the term diversity fails to address issues surrounding race/ 
ethnicity and does not account for the different histories, needs, interests, and 
issues affecting distinct groups of students on campus. With this in mind, 
one can see why it would be beneficial to use culture instead of diversity as 
the imperative to refocus assessment into a more inclusive endeavor. 

This paper draws from and expands on past definitions of culture to develop 
an understanding that culture should be thought of as: (1) the explicit 
elements that makes people identifiable to a specific group (s) including 
behaviors, practices, customs, roles, attitudes, appearance, expressions of 
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identity, language, housing region, heritage, race/ethnicity, rituals, religion; 
(2) the implicit elements that combine a group of people which include 
their beliefs, values, ethics, gender identity, sexual orientation, common 
experiences (e.g. military veterans and foster children), social identity; 
and (3) cognitive elements or the ways that the lived experiences of a 
group of people affect their acquisition of knowledge, behavior, cognition, 
communication, expression of knowledge, perceptions of self and others, 
work ethic, collaboration, and so on. The culturally relevant component 
involves assuring that the assessment process — beginning with student 
learning outcome statements and ending with improvements in student 
learning — is mindful of student differences and employs assessment methods 
appropriate for different student groups. Underlying the culturally relevant 
component is the focus on students — the importance of keeping students at 
the center, which requires their involvement at every step in the assessment 
process and builds upon their lived experience. 

In addition, it is important to understand the concept of intersectionality and 
its effect on culture. Traditionally, intersectionality is thought of in racial/ 
ethnic identity intersecting with class, gender, and sexual orientation to shape 
how people of color experience oppression (Solorzano & Yosso, 2002; Huber, 
2010; Cho, 1997). However, for purposes of this paper, intersectionality is 
the way that aspects of a persons identity cannot be fully separated from one 
another, play a central role in peoples’ experiences and making meaning of 
those experiences. This is related to Susan Jones and Marylu McEwen’s (2000) 
multiple dimensions of identity which treats a student’s identity as dynamic 
and changing depending on the relative contextual salience of other elements 
of ones identity (e.g. race/ethnicity, sexual orientation, gender, religion); 
with no single aspect of ones identity understood singularly, but only in 
relation to the other dimensions. For example, a White male that identifies 
as a member of the lesbian, gay, bisexual, transgender, queer, intersex, and 
asexual (LGBTQIA) community and practices Judaism is shaped by the 
intersectionality of these four elements. A Latina that is a single-mother from 
a low-socioeconomic background is shaped by the intersectionality of these 
elements. An undocumented English as a second language, first-generation 
student will experience college, acquire knowledge, and demonstrate 
knowledge differently than an international English as a second language 
first-generation student. The culture — the explicit, implicit, and cognitive 
elements — of the people in these examples shape their college experiences, 
and while one aspect of their culture may manifest itself more than another 
in specific contexts, they all affect the outcomes being assessed. 

Thinking of culture in the way that it is defined here can serve as a reference 
point for what to consider when engaging in assessment and developing/ 
choosing/implementing assessment tools and methods. Culture is by no 
means simple, and it is by no means easily definable. It is dependent on 
the context in which culture is discussed. Culture permeates the individual, 
group, entire institutions, countries, and continents; and at the same time 
the individuals that comprise cultural groups are multicultural through 
intersectionality. Perhaps Lang (1997) stated it best when he said “attempts 
at defining culture in a definite way are futile” (p. 389). However, developing 
an inclusive understanding of culture, and making it explicit that culture is 
much more than race/ ethnicity and affects students’ lives on multiple levels, 
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including learning and how they demonstrate learning, will help ensure 
culturally responsive assessment and increase the effectiveness and impact of 
learning outcomes assessment efforts. 

In addition to the term of culture, it is important to note the use of responsive 
to indicate “an action-based, urgent need to create contexts and curriculum 
that responds to the social, political, cultural, and educational needs of 
students; it is affirmative and seeks to identify and institutionalize practices 
that affirm indigenous and authentic cultural practices of students” (Khalifa, 
Gooden, & Davis, 2016, p. 1278). Students who experience validation from 
faculty and integrate academically and socially are more likely to persist and 
be successful (Karp, ffughes, & O’Gara, 2011). Assessment approaches 
and processes can help reinforce a sense of belonging or add to students’ 
belief that they do not belong because their learning or experiences are not 
deemed as valid or important. Susan ffeadden and Sarah McKay (2015) 
stress this point, arguing that student motivation is connected to students 
beliefs that they are able to do the work and have a sense of control over 
the work. For first-generation college goers and African American students 
“stereotypes about academic performance can turn into self-fulfilling 
prophecies. . .even feedback on papers can reinforce or foster learning. . .that 
students are cared about and respected as learners” (ffeadden & McKay, 
2015, p. 15). An environment focused on students’ unique learning interests 
and needs enables students to incorporate prior and everyday experiences in 
meaning construction (Land, ffannafin, & Oliver, 2012). Involvement with 
culture is also important as Cathleen Spinelli (2008) argues that there are a 
disproportionate number of students with cultural and linguistic differences 
that are misidentified as learning disabled. As a result, students are classified 
incorrectly, not academically challenged, and do not receive appropriate 
services. Spinelli (2008) further argues that when looking specifically at the 
case of English language learners, informal assessment provided a solution 
to the need of assessment of learning, but in a manner adaptable to language 
and cultural diversity, individual learning styles, and personal challenge 
while also informing instruction. 

Culturally responsive assessment is thus thought of as assessment that is 
mindful of the student populations the institution serves, using language 
that is appropriate for all students when developing learning outcomes, 
acknowledging students’ differences in the planning phases of an assessment 
effort, developing and/or using assessment tools that are appropriate for 
different students, and being intentional in using assessment results to improve 
learning for all students. Culturally responsive assessment involves being 
student-focused, which does not simply mean being mindful of students. 
Instead, being student-focused calls for student involvement throughout the 
entire assessment process including the development of learning outcome 
statements, assessment tool selection/development process, data collection 
and interpretation, and use of results. An essential aspect of maintaining 
focus on students is truly understanding the student population at the 
institution and/or level at which the assessment is being conducted. Once we 
understand who our students are we can begin to tailor assessment processes 
and materials to have the greatest impact for their learning. Institutions with 
high enrollment of traditionally underrepresented students have already 
begun tailoring their learning outcomes assessment approaches based on 
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the student populations that they serve (Montenegro & Jankowski, 2015; 
Nunley, Bers, & Manning, 2011; Baker, Jankowski, Provezis, & Kinzie, 
2012). Minority-Serving Institutions (MSIs) have been found to increase 
self-esteem, engagement, critical thinking skills, leadership skills and 
opportunities, and help the identity formation processes for traditionally 
underrepresented students; which helps increase students’ persistence 
through college (Conrad & Gasman, 2015; Del Rios & Leegwater, 2008; 
Conrad et al, 2013). The work at these institutions can serve as guideposts 
for the development of culturally responsive assessment practices. 

Student Learning Outcomes Statements 

Learning outcomes assessment as a process begins with developing learning 
outcome statements that clearly state what students should know and be 
able to demonstrate upon completion of a course, academic program, 
college, making use of student services, etc. To develop student learning 
outcomes statements using a cultural lens necessarily involves students in 
the development process. Poorly constructed learning outcomes make it 
difficult for students to demonstrate their learning for a myriad of reasons 
(e.g. not understanding what is expected of them, not understanding how 
the course/program is expected to contribute to their learning). In addition, 
it is students that will directly benefit from the feedback they receive as a 
result of assessment. Clarity of outcomes and curricular structure matters 
in general education (Gaston, 2015), assignment design (Winkelmes et al, 
2016), co-construction of knowledge for deep learning (Juvova et al, 2015; 
von Glasersfeld, 2005), and new course design models like competency-based 
education (Jobs for the Future, 2016). Further, in the National Research 
Council report, How People Learn , (2000) principles for designing learner- 
centered environments emphasized the importance of individual social and 
cultural contexts in learning. Such perspectives require different approaches 
to curricular design, teaching, and assessment, and squarely place learner 
preconceptions and experiences as an integral part of the learning process. 

Assessment is a field of alignment, and this also originates from learning 
outcomes statements. Hutchings (2016) defines alignment as “the linking of 
intended student learning outcomes with the processes and practices needed 
to foster those outcomes” (p. 5). Similarly to how academic programs, student 
services, and other institutional programs aim to align with and promote 
the mission of the college or university, learning outcomes statements of 
departments, programs, and courses should align with those of the institution. 
Outcome statements need to be culturally responsive because they align with 
assignments, evaluative criteria, and institutional and departmental goals. If 
outcome statements are not culturally responsive, then there are implications 
for various levels of the institution; not just for students. Learning outcome 
statements which are written to inform educational policy and practice, 
and are clear about expected proficiencies make it possible for programs, 
departments, institutions, and students to meet their goals (National 
Institute for Learning Outcomes Assessment, 2016). The language and 
operative verbs in learning outcome statements serve as a guide for students 
to understand departmental/program expectations, as well as understand 
how their educational experiences prepare them for their careers and lives 
after college (National Institute for Learning Outcomes Assessment, 2016). 
However, if learning outcomes statements are not written with attention to 
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cultural relevancy, then it becomes difficult to accurately infer the learning 
gains of different groups of students. 

Cliff Adelman (2015) speaks about the importance of being intentional and 
mindful of language when writing learning outcomes statements as this can 
lead to creating assignments that allow for genuine judgement of student 
achievement. However, taking this a step further and being mindful of how 
the language of learning outcomes statements might be appropriate for / 
inclusive of certain student groups but not others can lead to more holistic 
assessments. Flawed assessment designs may unintentionally skew scores 
for certain student populations and ensuring this does not happen begins 
with the writing of culturally responsive learning outcomes statements that 
consider students, their different ways of learning, and the diverse ways 
they demonstrate learning. One way to make statements more culturally 
responsive is to explicitly define terms and use scenarios or examples that 
are relatable to various student groups. A sample tool that incorporates these 
elements of being intentional and explicit in writing learning outcomes and 
clearly defining learning is the Degree Qualifications Profile (DQP) and 
Tuning process. The DQP was developed by Lumina Foundation (2014), 
and by coupling Tuning processes to it serves as a way to clearly outline 
what students know and should be able to do after attaining a degree (Ewell, 

2013). 

Traditionally, learning outcomes statements are written by and for faculty 
and administrators. As a result, faculty and administrators define the 
intended learning outcomes and what it looks like to demonstrate those 
outcomes. If, instead, we write learning outcome statements for and with 
students, then we increase the chances of students understanding what is 
expected of them. In addition, instead of students’ knowledge conforming 
to how we traditionally measure it, students would now have agency in how 
to demonstrate learning. This would result in learning outcomes, as well as 
the assessment process, becoming a more inclusive endeavor. 

Assessment Approaches 

There is a need for assessments that allow students to demonstrate their 
learning in various ways while also being transparent about the learning 
that is taking place, help students reflect on their learning experiences, and 
allow students to actively participate in the learning and assessment process. 
Course-level assessments such as culturally responsive rubrics, portfolios, and 
capstone projects can lead to more valid, appropriate, holistic, and formative 
assessment where results are more indicative of what all students can do 
or lead to more targeted improvements in teaching and learning. Rubrics, 
which help instructors gauge student learning, skills development, and 
acquisition of learning outcomes, provide criteria by which to assess whether 
or not the learning outcome was demonstrated. Rubrics, when they undergo 
a culturally conscious development process and are shared with students, 
can be a way to accurately assess learning for all students while allowing 
variation in how the learning is demonstrated. While rubrics are at times 
created by individual faculty members to fit the context of specific courses or 
programs, the Association of American Colleges and Universities (AAC&U) 
Valid Assessment of Learning in Undergraduate Education (VALUE) 
Rubrics serve as examples of rubrics for institutions to employ. In addition, 
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capstones — which can be entire courses or student projects — can also be 
an avenue through which the learning of students can be better gauged by 
allowing students to design their own projects in partnership with faculty. 
They provide “tangible, visible, self-explanatory evidence of exactly what 
students have and haven’t learned” (Suskie, 2004, p. 95). As comprehensive, 
culminating experiences, capstones allow students to demonstrate a wide- 
range of skills and knowledge that oftentimes draws from previous work, 
experiences, and learning that occurred throughout their coursework. 

Finally, portfolios offer a similar freedom for students to demonstrate 
their learning and provide a more holistic representation of what students 
know and can do. The use of portfolios provides students the option to 
select demonstrations and add commentary and reflection, furthering 
their agency in the process and selection of assessment evidence. Portfolios 
represent student work over time and demonstrate various forms of learning 
(Kuh et al, 2015; Banta, Griffin, Flateby, & Kahn, 2009) which may not 
be easily captured by other forms of assessment. Portfolios are “authentic 
assessment that draws on the work students do in regular course activities 
and assignments” and “reconnect assessment to the ongoing work of 
teaching and learning and to the work of faculty, raising the prospects for 
productive use” (Kuh et al, 2015, p. 36). Portfolios provide the opportunity 
to get students invested into the course beyond grade attainment, and help 
to deepen students’ educational experiences through allowing them to 
make connections between conceptual issues, theoretical knowledge, and 
real world experiences (Singer-Freeman & Bastone, 2016). Additionally, 
portfolios can be made available online. Eportfolios can be easily accessed 
by potential employers, as well as other institutions, which provides students 
in the job market or looking to transfer a means to easily demonstrate their 
knowledge and skills. Kuh et al (2015) mention a few of the advantages 
that portfolios have for assessment, including advancing student success, 
catalyzing change, and making learning more visible for students. These 
impacts can be furthered by applying a cultural lens when assessing student 
portfolios. By being mindful of how culture affects students’ meaning- 
making processes, cognition, and demonstrations of learning, we can better 
understand and appreciate the learning gains that students make. In fact, 
at the program-level, assessment approaches such as rubrics and portfolios 
are used more often than surveys and other approaches (Ewell, Paulson, & 
Kinzie, 2011). 

Use of Assessment Results 
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Implementing formative assessment methods means very little if assessment 
data are not used to inform learning at various levels of the institution or if it 
has no meaning to students to improve their own learning. The first step in 
creating change is analyzing the data by student populations. Disaggregating 
the data is instrumental in informing changes to higher education. While 
the data may tell a positive story about overall learning, disaggregation may 
yield the observation that first-generation students are struggling in a course, 
female students are making use of resources aimed at supporting their 
education at disproportionate rates, or Latinx students are not reaching the 
same institutional learning outcomes as other racial/ethnic groups. In either 
hypothetical case, disaggregating the data allows researchers, administrators, 
and practitioners to see themes that they otherwise would have missed and 
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could inform changes that would positively impact students’ education. 
In addition, disaggregation of assessment data should not only be used to 
uncover surface-level findings such as Latinx students excel at “ABC” while 
first-generation students struggle with “XYZ.” As Aydin Bal & Audrey Trainor 
(2016) state, “researchers must also include an examination of processes 
(e.g. the racialization of disability. . . and the institutional acts of exclusion 
based on ability differences) and institutions. . .that reproduce, regardless of 
intentionality, disparities” (p. 330-331). This means that disaggregating data 
should explore why the condition exists in the first place, and then be used 
to inform/develop possible solutions. 

In using assessment results, it is also useful to be mindful of our own 
assumptions. Similar to how a researcher’s bias cannot be fully removed from 
his/her/zer’s study and can either harm or enhance his/her/zer s research, so 
can the biases of faculty and staff affect assessment efforts and use of results. 
It is unrealistic and counterproductive for assessment professionals to think 
they are approaching their work from an impartial stance or to assume that 
the students being assessed also operate from an impartial stance. Failing to 
recognize how culture and our own experiences affect the assessment process 
can limit the impact of assessment. In discussing the need for faculty to be 
attentive to the changes in the institutions student population, Goldrick- 
Rab and Cook (2011) warn against comparing all students against the 
researchers subconscious idea of what students do/should do. Failing to be 
aware of our own biases or subconscious ideas and failing to disaggregate 
assessment data in a culturally responsive manner may cause the assessment 
endeavor to implement outdated norms as a means of comparison, which 
can misclassify certain students as underachievers, confusing, or outliers; and 
can also lead to the mistake of failing to connect the data to the actual lived 
experiences and realities of the students the institution serves (Goldrick-Rab 
& Cook, 2011). This can also lead to unintentionally reinforcing negative 
assumptions about certain student groups. Treating different racial/ethnic 
groups under an aggregate umbrella, as has been the recent case with the 
term “underrepresented minorities,” minimizes the voice of various groups 
and ignores their salient differences (Dowd & Bensimon, 2015) which 
impact their needs, experiences, learning, and demonstration of that 
learning. Finally, it would be worthwhile to connect assessment results to 
other campus assessment strategies. While certain data collection efforts 
on campus may seem unrelated, occurrences on campus seldom happen in 
isolation. Connecting different assessment efforts and resulting data sets can 
better inform issues related to student attrition, success, campus climate, 
pedagogy, and others (Hurtado & Halualani, 2014). 

Final Thoughts 
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Students’ college experiences are inseparable from other daily experiences such 
as those encountered at work, microaggressions endured on campus, family 
life, and employment. More often than not, students’ college experiences are 
affected by students’ own culture and cultural differences with faculty, staff, 
and peers. It has long been known that students of different backgrounds 
experience college differently and respond differently to similar situations, 
stimuli, experiences, requests, questions, etc. So, if we also know that students 
from different cultures who have similar education backgrounds respond 
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and perform significantly different, why would we design assessments, 
execute them, and then make changes based on assessment results without 
considering the cultural relevance of the assessment effort and analyze how 
the assessment might affect all students/benefit certain population (s) and 
hinder others? Why would we not include students in the assessment process 
to improve our approaches? 

The focus of assessment as a means to improve student learning is an agreed 
upon purpose of the work. As Kuh, Ikenberry, Jankowski, Cain, Ewell, 
Hutchings, and Kinzie (2015) state, “gathering information about collegiate 
outcomes has a practical goal: using it to improve both student learning 
and institutional performance” (p. 51); and “harnessing evidence of student 
learning, making it consequential in the improvement of student success 
and strengthened institutional performance is what matters” (p. 4). Yet, 
how consequential can assessment truly be when assessment approaches are 
minimally inclusive of our current student populations? Using assessment 
tools and approaches that work for the majority of students but are less 
mindful of students identifying with groups outside of the majority 
population places a significant portion of students at a disadvantage, leads 
to a decrease in the quality of education, creates a disconnect between 
students and the institution, and contributes to achievement gaps (Slee, 
2010; Sullivan, 2010; Qualls, 1998). Assessment that overlooks issues of 
diversity and equity contributes to inequalities in outcomes (Bal & Trainor, 
2016). The same can be said for assessment approaches that do not take into 
account students’ culture. 
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Students have different ways to demonstrate their knowledge and we need 
to use assessment metrics that appropriately elicit demonstrations of what 
students know. One example of the diverse ways students can demonstrate 
learning comes from Nick Sousanis’ (2015) published dissertation exploring 
how people construct knowledge. Instead of writing a typical manuscript, 
Sousanis demonstrated his knowledge in a graphic novel format. At times, 
the illustrations said more than the words on the page, and both pictures 
and words united to tell a powerful academic story. This way of presenting 
scholarly work, while unconventional in academia, is still a powerful 
demonstration of learning. Sousanis’ chosen method of demonstrating his 
knowledge on a specific topic is not wrong, it is just different. We undo 
boundaries through the awareness that “it is our [own] vision, and not what 
we are viewing, that is limited” (Sousanis, 2015, p. 42). How assessment is 
often operationalized or experienced by students has not moved to a position 
where it continuously regards students’ diverse methods of demonstrating 
knowledge as appropriate. Instead, different can often be marked as wrong. 

If assessment is about demonstrating learning, then we need to allow students 
the space to show their knowledge. Students are highly varied in customs, 
identity, and understanding, and it is all shaped by culture which affects 
learning; and thus, should affect how we measure learning. If assessment 
is done for improvement and with the goal of using the results to benefit 
student learning, then having outcome assessments that appropriately tell 
the stories of what students know and can do is of imperative importance. 
Our assessments approaches — how we assess and the process of assessment 
itself — should align with the students we have, empowering them with 
narratives to share and document their learning journey. 
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In summary, assessing students in the same way without paying attention 
to their differences works if students are all privy to the same educational 
opportunities, are all at the same academic standing, have similar experiences 
on campus, work through knowledge in similar fashion, understand questions 
in similar ways, and benefit from the same programs, pedagogical styles, 
support services, and interactions. However, we know this is not the case. 
“While absolute growth in the college-going population helped shape todays 
college milieu, compositional changes also impacted the college experience, 
turning it into a set of highly diverse experiences that led to very different 
outcomes” (Goldrick-Rab & Cook, 2011, p. 257). Sara Goldrick-Rab & 
Marjorie Cook (2011) continue to say that “as the student body grew more 
diverse, so did the kinds of colleges and universities serving them; at the same 
time, opportunities both expanded in number and became more distinct 
and disparate, reflecting and preserving key aspects of the inequality of 
opportunity and outcomes” (p. 255). Continuing to assess students as if there 
are no differences will only work to preserve key aspects of inequality and 
widen the achievement gap. It is no secret that there is a disparity between the 
academic attainment of students based on race/ethnicity (Bowen, Chingos, 
& McPherson, 2009; Condron, Tope, Steidl, & Freeman, 2013; Santiago, 
Galdeano, & Taylor, 2015; Carnevale & Strohl, 2013) and social class (Kezar, 
2011; Bowen, Chingos, & McPherson, 2009). We need to ask ourselves, is it 
that we want students to demonstrate their knowledge and skills or attainment 
of learning outcomes in a particular way, or that they demonstrate their 
learning? What is needed is not to help learners conform to the ways of higher 
education, thus reinforcing inequities and expectations based on ideologies 
the students may not ascribe to, but to empower students for success through 
intentional efforts to address inequality within our structures, create clear 
transparent pathways, and ensure that credits and credentials are awarded by 
demonstration of learning, in whatever form that may take. 



What is needed is not to help 
learners conform to the ways 
of higher education , thus 
reinforcing inequities and 
expectations based on ideologies 
the students may ascribe to, 
but to empower students for 
success through intentional 
efforts to address inequality 
within our structures, create 
clear transparent pathways, 
and ensure that credits and 
credentials are awarded by 
demonstration of learning, in 
whatever form that may take. 


National Institute for Learning Outcomes Assessment 


16 


References 


Adelman, C. (2015). To imagine a verb: The language and syntax of learning outcomes statements. Urbana, IL: University of 
Illinois and Indiana University National Institute for Learning Outcomes Assessment (NILOA). 

Aronson, B., & Laughter, J. (2016). The theory and practice of culturally relevant education: A synthesis of research across 
content areas. Review of Education Research , 86( 1), 163-206. 

Backes, B., & Velez, E. D. (2015). Who transfers and where do they go? Community college students in Florida. Washington, 
DC: National Center for Analysis of Longitudinal Data in Education Research (CALDER). 

Baker, G. R., Jankowski, N., Provezis, S. & Kinzie, J. (2012). Using assessment results: Promising practices of institutions 
that do it well. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes 
Assessment (NILOA). 

Bal, A., &Trainor, A. A. (2016). Culturally responsive experimental intervention studies: The development of a rubric for 
paradigm expansion. Review of Education Research, 86(2), 319-359. 

Banta, T. W., Griffin, M., Flateby, T. L., & Kahn, S. (2009, December). Three promising alternatives for assessing college 
students knowledge and skills. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning 
Outcomes Assessment. 

Benitez, M. (1998). Hispanic-serving institutions: Challenges and opportunities. New Directions for Higher Education, 

102 ( 2 ), 57 - 68 . 

Bowen, W. G., Chingos, M. M. & McPherson, M. S. (2009). Crossing the finish line: Completing college at Americas public 
universities. Princeton, NJ: Princeton University Press. 

Bransford, J. D., Brown, A. L., & Cocking, R. R. (Eds.). (2000). How people learn: Brain, mind, experience, and school. 
Washington, DC: National Academy Press. 

Carnevale, A. P., & Strohl, J. (20 1 3) . Separate and unequal: How higher education reinforces the intergenerational reproduction 
of White racial privilege. Washington, DC; Georgetown University, Georgetown Public Policy Institute, Center on 
Education and the Workforce. 

Check, E., & Ballard, K. (2014). Navigating emotional, intellectual, and physical violence directed toward LGBTQQ 
students and educators. Art Education, 67(3), 6-11. 

Cho, S. K. (1997). Converging stereotypes in racialized sexual harassment: Where the model minority meets Suzie Wong. 
In A. K. Wing, (Ed). Critical Race Feminism: A Reader (2nd. Ed). New York, NY: New York University Press. 

Chu, J. Y. (2016, October 21). The need to be culturally aware when advising students (essay). Inside Higher Ed. Retrieved 
from https://www.insidehighered.com/views/20 16/10/21/ need-be-culturally- aware- when-advising-students-essay. 

Condron, D. J., Tope, D., Steidl, C. R., & Freeman, K. J. (2013). Racial segregation and the Black/White achievement 
gap, 1992 to 2009. The Sociological Quarterly, 54, 130-157. 

Conrad, C., Gasman, M. Lundberg, T., Nguyen, T., Commodore, F., & A. C. Samayoa (2013). Using educational data 
to increase learning, retention, and degree attainment at Minority Serving Institutions. Philadelphia, PA: Penn Center for 
Minority Serving Institutions. Retrieved from http://www.gse.upenn.edu/pdf/cmsi/using_educational_data.pdf. 

Conrad, C., & Gasman, M. (2015). Educating a diverse nation: Lessons from Minority Serving Institutions. Cambridge, 
MA: Harvard University Press. 

Del Rios, M., & Leegwater, L. (2008). Increasing student success at minority serving institutions: Findings from the BEAMS 
project. Washington, DC: Institute for Higher Education Policy (IHEP). 

Dowd, A. G., & Bensimon, E. M. (2015). Engaging the “race question”: Accountability and equity in U.S. higher education. 
New York, NY: Teachers College at Columbia University. 

Ellis, L. M., & Chen, E. C. (2013). Negotiating identity development among undocumented immigrant college students: 
A grounded theory study. Journal of Counseling Psychology, 60(2), 251-264. 

Entwistle, N. (2001). Styles of learning and approaches to studying in higher education. Kybernetes, 30(516), 593-603. 

Ewell, P. T. (2013). The Lumina Degree Qualifications Profile: Implications for assessment. Urbana, IL: University of Illinois 
and Indiana University, National Institute for Learning Outcomes Assessment. 

Ewell, P. T., Paulson, K., & Kinzie, J. (2011). Down and in: Assessment practices at the program level. Urbana, IL: University 
of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA). 

National Institute for Learning Outcomes Assessment | 17 


Ferren, A. S., & Paris, D. C. (2015). Faculty leadership for integrative liberal learning. Washington, DC: Association of 
American Colleges and Universities. 

Froese-Germain, B., & McGahey, B. (2012). Optimizing conditions of teachers professional practice to support students with 
special educational needs. Ottawa, ON, Canada: Canadian Teacher’s Federation. Retrieved from http://files.eric.ed.gov/ 
fulltext/ED532565.pdf 

Gaston, P. L. (2015). General education transformed: How we can , why we must. Washington, DC: Association of American 
Colleges and Universities. 

Gay, G. (2011). Culturally responsive teaching: Theory, research, and practice (2nd ed.). New York, NY: Teacher College 
Press. 

Goldrick-Rab, S., & Cook, M. A. E. (2011). College students in changing contexts. In P. G. Altbach, P. J. Gumport, & 
R. O. Berdahl, (Eds) American higher education in the twenty -first century: Social, political, and economic challenges (3rd 
Ed.) pp. 254-278. Baltimore, MD: The John Hopkins University Press. 

Gosselin, J., & Gagne, A. (2014). Differentiated evaluation: An inclusive evaluation strategy aimed at promoting student 
engagement and student learning in undergraduate classrooms. Toronto, Canada: Higher Education Quality Council of 
Ontario. 

Grove, J. (2016, September, 29). Mature students ‘do better with non-written assessment’. Times Higher Education. 
Retrieved from https://www.timeshighereducation.com/ news/ mature-students-do-better-non-written-assessment. 

Headden, S., & McKay, S. (2015, July). Motivation matters: How new research can help teachers boost student engagement. 
Stanford, CA: Carnegie Foundation for the Advancement of Teaching. 

Huber, L. P. (2010). Using Latina/o critical race theory (LatCrit) and racist nativism to explore intersectionality in the 
educational experiences of undocumented Chicana college students. Educational Foundations, 24, 77-96. 

Hurtado, S., & Halualani, R. (2014). Diversity assessment, accountability, and action: Going beyond the numbers. 
Diversity & Democracy, 17(4), 8-11. 

Hutchings, P. (2016) . Aligning educational outcomes and practices. Urbana, IL: University of Illinois and Indiana University, 
National Institute for Learning Outcomes Assessment (NILOA). 

Jobs for the Future. (2016, February). Expanding competency-based education for all learners. Washington, DC: Author. 

Jones, S. (2015). The game changers: Strategies to boost college completion and close attainment gaps. Change: The 
Magazine of Higher Learning, 47(2), 24-29. 

Jones, S. R., & McEwen, M. K. (2000). A conceptual model of multiple dimensions of identity. Journal of College Student 
Development, 41, 405-414. 

Juvova, A., Chudy, S., Neumeister, P., Plischke, J., & Kvinotova, J. (2015). Reflection of constructivist theories in current 
educational practice. Universal Journal of Educational Research, 3(5), 345-349. 

Karp, M. M., Hughes, K. L., & O’Gara, L. (2010). An exploration of Tinto’s integration framework for community 
college student s. Journal of College Student Retention: Research, Theory & Practice, 12(\), 69-86. 

Kezar, A. (2011). Recognizing and serving low-income students in higher education: An examination of institutional policies, 
practices, and culture. New York, NY: Routledge. 

Khalifa, M. A., Gooden, M. A., & Davis, J. E. (2016). Culturally responsive school leadership: A synthesis of the literature. 
Review of Educational Research, 86(4) , 1272-1311. 

Kim, E., & Diaz, J. (2013). Undocumented students and higher education. ASHE Higher Education Report, 38(6), 77-90. 

Kuh, G. D., Ikenberry, S. O., Jankowski, N. A., Cain, T. R., Ewell, P. T., Hutchings, P., & Kinzie, J. (2015). Using evidence 
of student learning to improve higher education. San Francisco, CA: Jossey-Bass. 

Kuh, G. D., Jankowski, N., Ikenberry, S. O., & Kinzie, J. (2014). Knowing what students know and can do: The current 
state of student learning outcomes assessment in US colleges and universities. Urbana, IL: University of Illinois and Indiana 
University, National Institute for Learning Outcomes Assessment (NILOA). 

Ladson-Billings, G. (1995a). Toward a theory of culturally relevant pedagogy. American Educational Research Journal, 
32( 3), 465-491. 

Ladson-Billings, G. (1995b). But that’s just good teaching! The case for culturally relevant pedagogy. Theory Into Practice, 

34(3), 159-165. 


National Institute for Learning Outcomes Assessment | 18 


Land, S. M., Hannafin, M. J., & Oliver, K. (2012). Student-centered learning environments: Foundations, assumptions and 
design. In D. Jonassen, & S. Land (Eds.). Theoretical foundations of learning environments (2nd ed.) pp. 3-26. New York, 
NY: Routledge. 

Lang, A. (1997). Thinking rich as well as simple: Boeschs cultural psychology in semiotic perspective. Culture & Psychology, 
3, 383-394. 

Lara, R., & Wood, J. L. (2015). Impact of non-cognitive behaviors on student engagement for Latino male students attending 
a designated two-year Hispanic-serving institution. In J. P. Mendez, F. A. Bonner II, J. Mendez-Negrete, & R. T. Palmer 
(Eds) Hispanic-serving institutions in American higher education: Their origin , and present and future challenges. Sterling, VA: 
Stylus Publishing, LLC. 

Li, X., & Carroll, C. D. (2007). Characteristics of minority-serving institutions and minority undergraduates enrolled in these 
institutions. Washington, DC: National Center for Education Statistics, Institute of Education Sciences, U.S. Department 
of Education. 

Lumina Foundation (2014). The Degree Qualifications Profile. Indianapolis, IN: Author. 

Macqueen, E. (2012). Studying the experiences of non-traditional students in teacher education. Sydney, Australia: Joint AARE 
APERA International Conference. Retrieved from http://files.eric.ed.gov/fulltext/ED544492.pdf. 

Mallory, A. (2009). Access and equity for all students: Meeting the needs ofLGBT students. California Postsecondary Education 
Commission. Retrieved from http://files.eric.ed.gov/fulltext/ED507508.pdf. 

McKenzie, K. B., & Scheurich, J. J. (2004). Equity traps: A useful construct for preparing principals to lead schools that are 
successful with racially diverse students. Educational Administration Quarterly ; 40, 601-632. 

Meents-DeCaigny, E., & Sanders, L. (2015). Framing research and assessment of co-curricular learning. New Directions for 
Institutional Research, 1 64, 15-25. 

Methvin, P., & Markham, P. N. (2015). Turning the page: Addressing the challenge of remediation. Change: The Magazine 
of Higher Learning, 47(4), 50-56. 

Miller, R., Bradbury, J., & Pedley, J. (1998). Academic performance of first and second language students: Disadvantages and 
underpreparedness. South African Journal of Science, 95, 30-34. 

Montenegro, E., & Jankowski, N. A. (2015). Focused on what matters: Assessment of student learning outcomes at minority- 
serving institutions. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes 
Assessment (NILOA). 

National Institute for Learning Outcomes Assessment. (2016, May). Higher education quality: Why documenting learning 
matters. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment. 

Newman, L. E., Carpenter, S., Grawe, N., & Jaret-McKinstry, S. (2014). Creating a culture conducive to integrative learning. 
Peer Review, 16(4), 14-15. 

Nunley, C., Bers, T, & Manning, T. (2011, July). Learning outcomes assessment in community colleges. Urbana, IL: University 
of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA). 

Pascarella, E. T., &Terenzini, P. T. (2005). How college affects students. San Francisco, CA: Jossey-Bass. 

Perez, W. (2010). Higher education access for undocumented students: Recommendations for counseling professionals. 
Journal of College Admission, 206, 32-35. 

Person, A. E., Goble, L., & Bruch, J. (2014, April). Developing competency-based program models in three community colleges. 
Oakland, CA: Mathematica Policy Research. 

Qualls, A. L. (1998). Culturally responsive assessment: Development strategies and validity issues. The Journal of Negro 
Education, 67(3), 296-301. 

Rankin, S., Weber, G., Blumenfeld, W., & Frazer, S. (2010). 2010 state of higher education for lesbian, gay, bisexual, and 
transgender people. Charlotte, NC; Campus Pride. 

Santiago, D. A., Galdeano, E. C., & Taylor, M. (2015). The condition of Latinos in education: 2015 factbook. Washington, 
DC; Excelencia in Education. 

Schuh, J. H. (2009). Assessment methods for student affairs. San Francisco, CA: Jossey-Bass. 


National Institute for Learning Outcomes Assessment | 19 


Schuh, J. H., Jones, S. R., Harper, S. R., & Associates. (2011). Student services: A handbook for the profession (5th Ed). San 
Francisco, CA: Jossey-Bass. 

Sedlacek, W. E. (1994). Issues in advancing diversity through assessment. Journal of Counseling & Development, 72 , 549-553. 

Shapiro, D., Dundar, A., Chen, J., Ziskin, M., Park, E., Torres, V., & Chiang, Y. (2012). Completing college: A national view 
of student attainment rates. Herndon, VA: National Student Clearinghouse Research Center. 

Singer-Freeman, K. & Bastone, L. (2016, July). Pedagogical choices make large classes feel small. (Occasional Paper No. 27). 
Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA). 

Singleton, G. E. (2012). More courageous conversations about race. Thousand Oaks, CA: Corwin. 

Slee, J. (2010). A systemic approach to culturally responsive assessment practices and evaluation. Higher Education Quarterly, 

64(3), 246-260. 

Solorzano, D. G., & Yosso, T. J. (2002). Critical race methodology: Counter-storytelling as an analytical framework for 
education research. Qualitative Inquiry, 5(1), 23-44. 

Sousanis, N. (2015). Unflattening. Cambridge, MA: Harvard University Press. 

Spinelli, C. G. (2008). Addressing the issue of cultural and linguistic diversity and assessment: Informal evaluation measures 
for English language learners. Reading & Writing Quarterly, 24(1), 101-118. 

Strange, C. C., & Banning, J. H. (2015). Designing for learning: Creating campus environments for student success (2nd Ed.). 
San Francisco, CA: Jossey-Bass. 

Sullivan, A. L. (2010). Preventing disproportionality: A framework for culturally responsive assessment. Communique, 39(3), 
24-26. 

Suskie, L. (2004). Assessing student learning: A common sense guide. Boston, MA: Anker Publishing Company. 

Tobolowski, B. F., & Cox, B. E. (2012). Rationalizing neglect: An institutional response to transfer students. Journal of 
Higher Education, 83(3), 389-410. 

Toomey Zimmerman, H., & Bell, P. (2012). Everyday expertise: Learning within and across formal and informal settings. 
In D, Jonassen & S. Land (Eds.). Theoretical foundations of learning environments (2nd ed.) pp. 224-241. New York, NY: 
Routledge. 

von Glasersfeld, E. (2005). Instroduction: Aspects of constructivism. In C.T. Fosnot (Ed.), Constructivism: Theory, Perspectives, 
and practice (2nd ed.), pp. 3-7. New York, NY: Teachers College Press. 

Winkelmes, M., Bernacki, M., Butler, J., Zochowski, M., Golanics, J., & Harriss Weavil, K. (2016). A teaching intervention 
that increases underserved college students’ success. Peer Review, 75(1-2), 31-36. 


National Institute for Learning Outcomes Assessment | 20 


NILOA National Advisory Panel 


James Anderson 

Interim Dean 

Edward William and Jane Marr Gutsgell 
Professor 

University of Illinois Urb ana-Champaign 

Wallace Boston 

CEO 

American Public University System 

Molly Corbett Broad 

President 

American Council on Education 

Judith Eaton 

President 

Council for Higher Education Accreditation 

Richard Ekman 

President 

Council of Independent Colleges 

Keston Fulcher 

Director of the Center for Assessment and 
Research Studies 

Associate Professor, Graduate Psychology 
James Madison University 

Paul Gaston, III 

Trustees Professor 
Kent State University 

Mildred Garcia 

President 

California State University, 

Fullerton 

Susan Johnston 

Executive Vice President 
Association of Governing Boards 

Norman Jones 

Professor 

Utah State University 

Peggy Maki 

Higher Education Consultant 

George Mehafly 

Vice President for 
Academic Leadership and Change 
American Association of State Colleges and 
Universities 


George Pernsteiner 

President 

State Higher Education Executive Officers 
Association 

Mary Ellen Petrisko 

President 

WiSC Senior College and University 
Commission 

Kent Phillippe 

Associate Vice President, Research and 
Student Success 

American Association of Community Colleges 

Robert Sheets 

Research Professor 

George Washington Institute of Public Policy 

Ralph Wolff 

Founder and President 

The Quality ASsurance Commons for Higher 

and Postsecondary Education 

Ex-Officio Members 

Peter Ewell 

President Emeritus 

National Center for Higher Education 
Management Systems 

Stanley Ikenberry 

President Emeritus and Regent Professor 
University of Illinois 

Natasha Jankowski 

Director, NILOA 
Research Assistant Professor 

George Kuh 

Founding Director, National Institute for 
Learning Outcomes Assessment 
Adjunct Research Professor, University of 
Illinois Urb ana-Champaign 
Chancellor’s Professor of Higher Education 
Emeritus, Indiana University 

Paul Lingenfelter 

President Emeritus 

State Higher Education Executive Officers 


Lynn Pasquerella 

President 

Association of American Colleges & 
Universities 



NILOA Mission 

NILOA’s primary objective is to 
discover and disseminate ways that 
academic programs and institutions 
can productively use assessment data 
internally to inform and strengthen 
undergraduate education, and exter- 
nally to communicate with policy 
makers, families and other stake- 
holders. 

NILOA Occasional Paper Series 

NILOA Occasional Papers 
are commissioned to examine 
contemporary issues that will inform 
the academic community of the 
current state-of-the art of assessing 
learning outcomes in American higher 
education. The authors are asked to 
write for a general audience in order 
to provide comprehensive, accurate 
information about how institutions and 
other organizations can become more 
proficient at assessing and reporting 
student learning outcomes for the 
purposes of improving student learning 
and responsibly fulfilling expectations 
for transparency and accountability 
to policy makers and other external 
audiences. 


Comments and questions about this 
paper should be sent to: 
niloa@education.illinois.edu 


National Institute for Learning Outcomes Assessment 


21 


About NILOA 


• The National Institute for Learning Outcomes Assessment (NILOA) was estab- 
lished in December 2008. 

• NILOA is co-located at the University of Illinois and Indiana 
University. 

• The NILOA website contains free assessment resources and can be found at http:// 
www.learningoutcomesassessment.org/. 

• The NILOA research team has scanned institutional websites, surveyed chief 
academic officers, and commissioned a series of occasional papers. 

• NILOA’s Founding Director, George Kuh, founded the National Survey for 
Student Engagement (NSSE). 

• The other co-principal investigator for NILOA, Stanley Ikenberry, was president 
of the University of Illinois from 1979 to 1995 and of the American Council of 
Education from 1996 to 2001. 

NILOA Staff 

NATIONAL INSTITUTE FOR LEARNING OUTCOMES ASSESSMENT 
Stanley Ikenberry, Co-Principal Investigator 

George Kuh, Founding Director, Senior Scholar, and Co-Principal Investigator 

Natasha Jankowski, Director 

Gianina Baker, Assistant Director 

Filip Przybysz, Communications Coordinator 

Katie Schultz, Project Manager 

Peter Ewell, Senior Scholar 

Pat Hutchings, Senior Scholar 

Jillian Kinzie, Senior Scholar 

Paul Lingenfelter, Senior Scholar 

David Marshall, Senior Scholar 

Erick Montenegro, Research Analyst 

Verna F. Orr, Research Analyst 

Anthony B. Sullers, Jr., Research Analyst 

Emily Teitelbaum, Research Analyst 

Terry Vaughan III, Research Analyst 

Karie Brown-Tess, Research Analyst 

NILOA Sponsors 

Lumina Foundation for Education 
University of Illinois, College of Education 


Produced by Creative Services | Public Affairs at the University of Illinois for NILOA. 1 0.032 


National Institute for Learning Outcomes Assessment | 22 



n success ingenuity intellect curiosity challenge create achievement knowledge accountability connection self-reflection educate action understan 
cate curiosity challenge create achievement connection self-reflection understand communicate listen learn access quality action educate actio 
nd communicate listen learn action understand communicate listen learn access quality innovation success ingenuity intellect curiosity challeng 
[ e accountability connection access quality self-reflection curiosity challenge create achievement learn access quality innovation success ingenuit 
ction educate action understand intellect knowledge accountability connection self-reflection educate action understand knowledge accountabilit 
)n self-reflection educate action understand communicate listen learn access quality innovation success ingenuity intellect curiosity challeng 
)n knowledge accountability connection self-reflection educate action understand communicate listen learn access quality innovation succes 
challenge create achievement connection self-reflection educate action understand connection self-reflection understand communicate listen lear 
uality action create achievement connection self-reflection educate action understand communicate listen learn access quality innovation succes 
action communicate listen learn access quality action educate action understand communicate educate innovation success self-reflection knowledg 
bility communicate listen learn achievement connection self-reflection educate action understand communicate listen learn access quality innovatio 
ingenuity intellect access quality innovation success self-reflection curiosity challenge create achievement connection self-reflection understan 
action understand communicate listen learn action understand communicate listen learn access quality innovation success ingenuity curiosit 
5 create achievement connection self-reflection understand communicate listen learn access quality action create achievement connection sel 
1 educate action understand communicate listen learn access quality innovation success educate action communicate listen learn access qualit 
ucate action understand create achievement connection self-reflection understand communicate listen learn access quality action create achievemer 
)n self-reflection educate action understand communicate listen communicate educate innovation success self-reflection knowledge accountabilit 
)n self-reflection educate action understand communicate listen learn access quality innovation ingenuity intellect connection self-reflectio 
nd communicate listen learn access quality action create achievement connection self-reflection educate action understand communicate listen lear 


National Institute for Learning Outcomes Assessment 


For more information, please contact: 

National Institute for Learning Outcomes Assessment (NILOA) 
University of Illinois at Urbana-Champaign 
360 Education Building 
Champaign, IL 61820 

learningoutcomesassessment.org 
niloa@education.illinois.edu 
Phone: 21 7.244.21 55 


